Benefit of stereoscopic volume rendering for the identification of pediatric pulmonary vein stenosis from CT angiography

The use of three-dimensional (3D) technologies in medical practice is increasing; however, its use is largely untested. One 3D technology, stereoscopic volume-rendered 3D display, can improve depth perception. Pulmonary vein stenosis (PVS) is a rare cardiovascular pathology, often diagnosed by computed tomography (CT), where volume rendering may be useful. Depth cues may be lost when volume rendered CT is displayed on regular screens instead of 3D displays. The objective of this study was to determine whether the 3D stereoscopic display of volume-rendered CT improved perception compared to standard monoscopic display, as measured by PVS diagnosis. CT angiograms (CTAs) from 18 pediatric patients aged 3 weeks to 2 years were volume rendered and displayed with and without stereoscopic display. Patients had 0 to 4 pulmonary vein stenoses. Participants viewed the CTAs in 2 groups with half on monoscopic and half on stereoscopic display and the converse a minimum of 2 weeks later, and their diagnoses were recorded. A total of 24 study participants, comprised of experienced staff cardiologists, cardiovascular surgeons and radiologists, and their trainees viewed the CTAs and assessed the presence and location of PVS. Cases were classified as simple (2 or fewer lesions) or complex (3 or more lesions). Overall, there were fewer type 2 errors in diagnosis for stereoscopic display than standard display, an insignificant difference (p = 0.095). There was a significant decrease in type 2 errors for complex multiple lesion cases (≥3) vs simpler cases (p = 0.027) and improvement in localization of pulmonary veins (p = 0.011). Subjectively, 70% of participants stated that stereoscopy was helpful in the identification of PVS. The stereoscopic display did not result in significantly decreased errors in PVS diagnosis but was helpful for more complex cases.


Introduction
Volume rendering in medical imaging refers to a visual reconstruction technique which renders an image by sampling voxels of a 3D data set, such as computed tomography (CT) or magnetic resonance imaging (MRI) volumetric data [1]. Despite its 3D appearance, the absence of stereopsis, the perception of depth resulting from combining visual stimuli from both eyes, results in the loss of depth cues when volume-rendered images are displayed using 2D screens, the current standard in medical imaging. In contrast, stereoscopic displays allow for stereopsis and convey depth information through binocular vision. Stereoscopy has been shown to enhance the perception of contrast [2]. In the medical context, the stereoscopic display has been shown to improve visual-spatial tasks and visualization [3,4].
There are many newer approaches to the 3D display of congenital heart disease which have been shown to be beneficial, including 3D printing, mixed reality, virtual reality, and augmented reality [5,6,7]. With recent advances in printing technology, 3D printing has been advocated as a useful method of displaying cardiac anatomy for procedural planning [8,9]. While 3D printing has garnered numerous publications, stereoscopic displays are less commonly studied. Stereoscopic display advantages include the requirement for only a 3D display screen or projector, and goggles, and viewing by multiple users at the same time. In contrast, mixed reality, augmented reality and virtual reality displays are mostly limited to one user at a time. Stereoscopic display has been studied in liver CT for surgical planning [10], digital mammography cancer screening, [11], multimodal brain imaging [12] and 3D echocardiography for congenital heart disease diagnosis [13]. Although 3D modalities are widespread, there is relatively little objective information on the potential additive improvement of 3D stereoscopic display to diagnosis in medical imaging, particularly cardiovascular applications.
Pulmonary vein stenosis (PVS) is a significant cardiovascular pathology in which single or multiple pulmonary veins can become narrowed or obstructed. Echocardiography is the firstline modality for evaluation of patients with suspected PVS, however, an additional form of imaging is commonly employed for confirmation prior to intervention, because of limited sonographic visibility. While CT or MRI can be used, CT has higher spatial resolution and excellent diagnostic performance [14,15]. CT volume rendering improved diagnostic accuracy and confidence in pediatric thoracic vascular disorders [16,17,18] and for pediatric proximal pulmonary vein stenosis [19]. Considering the added benefit of volume rendering for this disease, its variability in presentation, and its reliance on CT for diagnosis, PVS is an ideal condition for investigating the added benefit of stereoscopic display of the volume-rendered images.
The objective of this study was to compare the effect of flat screen conventional display with 3D stereoscopic display of volume-rendered CT on physician perception, using pediatric pulmonary vein stenosis as a test diagnosis.

Methods
This study was a retrospective observational study using retrospectively acquired CT images and physician volunteers as observers. The study was approved by the University of Alberta Human Research Ethics Board.

CT images and post-processing
CT images of 22 pediatric patients scanned for pulmonary vein stenosis were acquired retrospectively and consecutively from July 2013 to May 2015 from the radiology pictorial archiving and communications system (PACS). Inclusion criteria were all pediatric patients who had at least 1 year of clinical, surgical or interventional cardiology follow up and acceptable CT image quality. Patient studies were excluded if image quality was poor or there was no follow up or if a prior scan had already been enrolled. Patient ages ranged from 3 weeks to 2 years with a mean of 8 months; there were 8 females and 14 males, weighing 2.9 to 12 kg. The patients were scanned on a Siemens Somatom FLASH scanner, scanned at 80 to 100 kV, 20 mAs, CTDI 0.75 to 1.68 mGy.
The axial CT DICOM images were segmented using Aquarius software (TeraRecon, Inc., San Mateo, California). Diagnoses were confirmed by an experienced pediatric cardiac radiologist with access to all CT images (multiplanar reformats, maximal intensity projections, volume rendering) and patient charts (operative findings, follow-up imaging and clinical data). Manual segmentation was used to remove the adjacent structures, primarily the bones, lungs, aorta and larger pulmonary arteries to show the pulmonary veins and left atrium. A standard CT volume rendering transfer function was employed for all data sets for display on our homegrown display system. The volume rendered images were placed on the stereoscopic system for display on flat screen in either 2D or 3D stereoscopic modes (Fig 1). Only the volume rendered datum was presented, and there was no access to standard axial images or multiplanar reformats (Fig 1).
Of the 18 datasets used for the analysis, there were a total of 31 pulmonary vein stenoses or occlusions. There were 4 normal patients, 4 patients with 1 pulmonary vein stenosis or occlusion, 4 patients with 2 pulmonary vein stenoses or occlusions, 5 patients with 3 pulmonary vein stenoses or occlusions and 1 patient with 4 pulmonary vein stenoses or occlusions. Images were stratified into pulmonary vein stenosis (14 patients) and normal (4 patients) and then randomly divided into 2 groups of 9 patients each (9 for each set A/B and set C/D). Sets A and B had the same 9 test cases, and sets C and D had the same 9 test cases (Fig 2). The remaining 4 non-test patient datasets were randomly distributed to one of set A, B, C, or D, thus creating 4 slightly differing groups. The 4 randomly distributed datasets were used for the participant sessions, however, results were not counted, so that the test sets A and B were identical, and test sets C and D were identical as far as the 9 patients in each group used for analysis. The non-test cases were added to decrease the participant's memory of similarity between test sets. Cases were classified as simple (2 or fewer lesions) or complex (3 or more lesions), and distributed evenly within the groups.

Volume rendering and display
All images were displayed using a Christie Mirage S+6K DLP projector and Starglass screen (58"x78") for both monoscopic and stereoscopic viewing. The direct volume rendering technique was used for displaying the volumetric CT data [1]. Viewing software was implemented in Python programming language using the Visualization Toolkit (VTK) open-source software module (Kitware Inc., Clifton Park, New York, USA) [20], and Qt graphical user interface library (The Qt Company, Espoo, Finland). The scans were rendered with an NVI-DIA Quadro K5200 graphics processor using a ray casting technique, which performs an image-order rendering. All the images were rendered using the same customized color and opacity transfer functions, which were edited by an experienced radiologist, and could not be altered for this study. In the opacity transfer function, higher opacity was assigned for intensity values corresponding to cardiac tissues, and lower opacity was for intensities corresponding to other organs such as lungs to make them transparent. The color transfer function was edited interactively to enhance the aesthetics of cardiac tissues and vessels. The refresh rate for the project was set to 90 Hz, and no lag in rendering was observed during the interactions such as rotation and scaling. 3D scans were viewed with active shutter glasses, and users manipulated the images using a mouse.

Participants
Participants were recruited from the departments/ divisions of radiology, pediatric cardiology, pediatric critical care, and cardiovascular surgery. All participants were physicians. Trainees were either residents or fellows. Subject selection assumed appropriate anatomic and pathologic knowledge of pulmonary vein stenosis. All participants signed written informed consent for the study.

2D vs 3D image evaluation study
The volunteer study involved a pretest session and 2 sessions for each participant, with the second session occurring at least 2 weeks after the first. In the pretest session, a random-dot stereogram was employed to ensure that participants had stereoscopic perception [21]. A training case was provided to teach basic anatomic positions and to become familiar with using the software to manipulate the images.
For the first testing session, participants were randomly assigned to one of 2 viewing orders: either set A with monoscopic viewing followed by C with stereoscopic viewing or B with stereoscopic viewing followed by D with monoscopic viewing (Fig 3). Participants were not aware of the number of normal, pathological or unscored cases. A participant questionnaire

PLOS DIGITAL HEALTH
was administered during the session and included identification of specific pulmonary veins by pointing with a cursor, in addition to identifying pulmonary vein stenosis. Sessions were video-recorded and later reviewed to confirm the accuracy of scoring.
In the second testing session, participants viewed the images in the converse display mode to the first session, viewing the same cases but in the alternate viewing display to the first session (Fig 3). The same questionnaire was administered as in the first session and the session recorded. Participants were not told that 18 of the cases viewed were the same as in the first session but with the converse viewing method.
Errors in identifying pulmonary vein stenosis were reported as type 1 (false positive) or type 2 (false negative) errors. Errors were also recorded if a participant pointed to the wrong vein when asked to localize a specific pulmonary vein. The time that each observer spent on each case was recorded, except for the first case of each set; observers needed significantly more time on the first case of each session to become accustomed to the system.

Statistics
Statistical analysis was performed using the R programming language [22]. The ggplot2 data visualization package for R was used [23]. In addition, the reshape 2 package for R was also used to aid in data visualization [24]. Mann Whitney two-tailed tests were performed to determine the statistical significance of time spent per case between stereoscopy and monoscopy for each participant because of the absence of 3 time data points. Wilcoxon signed rank two-tailed tests were performed in order to determine statistical significance for type 1, 2 and localization errors between stereoscopy and monoscopy for each participant. P values were also calculated for the other combinations of cases and participants for monoscopic vs stereoscopic views. An alpha level of p value less than 0.05 was set as the threshold for statistical significance.

Participant questionnaire
The following questions were posed: • Name the vessel(s) with pulmonary vein stenosis. Point to it.
• Point to the _________ pulmonary vein (specified vessel).
• Do you feel that the addition of 3D visualization aided in your skill performance?

Results
There were 27 participants, comprised of 15 staff physicians and surgeons, (8 radiologists, 4 cardiologists and 3 cardiac surgeons) and 12 trainees (6 radiology trainees, 5 cardiology trainees, and 1 cardiac surgery trainee). All participants were able to appreciate stereopsis. 3 participants (2 staff physicians and 1 trainee) did not complete the second session; therefore, their results were removed from the study, for a total of 24 participants.
Time spent viewing each case for monoscopic display was a mean of 65.4 +/-38.9 seconds, median 57.3 seconds, range 12.5 to 260.2 seconds (S1 Table). For stereoscopic viewing, the mean time was 66.9 +/-35.4 seconds, median 60.1 seconds, range 11.3 to 257.5 seconds (S1 Table). The time difference between the two types of displays was insignificant (Mann Whitney U1 = 69170.5, U2 = 77133.5, n1 = 381, n2 = 384, p = 0.10, two-tailed). Time was not recorded for 3 out of the 768 cases observed as participants experienced outside interruptions during their sessions.
A majority of participants (16 of 23, 69.5%), stated that stereoscopy helped them in the identification of PVS, with greater confidence in the diagnosis. One participant recorded no response to this question.

Discussion
This study represents one of the few objective studies measuring the effect of 3D stereoscopic display on clinician perception in cardiovascular imaging, in this case, for the diagnosis of pulmonary vein stenosis. While there has been a previous study showing an advantage for 3D volume rendering over standard multiplanar reconstruction for this disease [19], no study has assessed the advantage of 3D stereoscopic display for volume-rendered 3D images in assessment of pulmonary vein stenosis. Our study has shown improved diagnostic accuracy and confidence in recognition of pulmonary vein stenosis, and improvement in anatomic localization of pulmonary veins with stereoscopic display.
There was a trend toward improved accuracy of diagnosis for all pulmonary vein stenosis cases, which was not statistically significant, and likely reflected the large number of type 2 errors for complex cases. There was an improvement in accuracy and confidence of diagnosis of pulmonary vein stenosis with fewer type 2 errors for the more complex diagnostic cases with multiple lesions. These results are consistent with other studies that show an improvement in diagnosis using stereoscopic 3D displays [13,25]. Liu showed that the accuracy of detection of cerebral aneurysms was the same (100%) for stereoscopy and conventional CT workstation display for aneurysms over 3 mm; however, it improved with aneurysms less than 3 mm (94% for stereoscopy and 82% for conventional workstation) [25]. Harake et al found that clinicians preferred viewing 3D echocardiography images of both simple and complex lesions in stereoscopic display, as evidenced by more time and interaction spent viewing stereoscopic display when both displays were equally available for viewing [13]. A larger study may have helped to distinguish whether there was an improvement in perception for all types of pulmonary vein stenosis or whether the advantage occurs primarily with complex pathology. Further study would also be required to determine whether the perception advantage for 3D stereoscopic display is dependent on the type of 3D imaging and the specific pathology.
There was an improvement in localization of pulmonary veins with stereoscopic display. This localization task tested the participants' spatial orientation. In diagnosis, correctly locating pathology is as important as recognizing the pathology. If participants were disoriented, they may have recognized pulmonary vein pathology but incorrectly identified the location, resulting in more type 2 errors. The decision to separate the PVS cases into simple and complex was not based on a standard classification. However based on our observation, it became much more difficult for participants to track the position of stenoses spatially as the number of stenoses increased. This assertion is supported by the fact that 63% of type 2 errors with monoscopic display and 59% of type 2 errors in stereoscopic display were attributed to complex cases, while complex cases only comprised 33% of the overall number of cases.
There was a subjective preference for stereoscopy in our study, similar to Harake et al [13]. The preference was not quantitated, and the subjective impression is difficult to measure. This preference is also supported by a few studies that show that 3D printing, which is more widely studied for cardiothoracic surgery, aids in the understanding of spatial relationships, although these studies also cannot quantitate that preference [7,26,27]. Some studies have cited a greater preference on the part of trainees for 3D printing in pre-operative planning, although preference was not quantitated [28].
Results from trainees were compared to staff physicians and surgeons in this study, but there was no significant difference in error rates between these two groups. In general, staff physicians and surgeons showed a larger improvement with stereoscopy than monoscopy compared to the trainees, although this difference was not statistically significant. This would be an area for further study, to determine whether there are differences between experienced and inexperienced viewers for stereoscopy and monoscopy. The role of 3D printing has been more studied, and like stereotactic visualization, some studies have cited a greater preference on the part of trainees for viewing 3D printing in pre-operative planning [28]. For surgical training, virtual reality training was not as optimal as wet lab simulation but has been shown to present a reasonable alternative to wet lab simulation [29].
The ground truth was determined by a single individual which could bias the study results. However, the individual had access to all follow up surgical, catheterization, imaging and clinical chart data to confirm diagnoses. While confirming ground truth by a team of individuals would have been useful, the follow up data was likely more objective as a measure of ground truth.
A limitation of our study was the use of only volume rendered images to make a diagnosis of pulmonary vein stenosis. Usually in clinical practice, physicians view conventional multiplanar reconstructions extensively and therefore may have made more type 2 errors than usual. The use of volume rendered images only may also account for the similar performance of staff physicians to trainees, as staff physicians are accustomed to using typical multiplanar images for this diagnosis. Participants were deliberately not shown the conventional multiplanar reconstructions so that they would not be biased by their impression from the 2D planar reconstructions. By showing only volume rendered images, we attempted to evaluate only the effect of the different display modes on accuracy of diagnosis, not the optimal diagnostic pathway in a real world clinical situation. We are not suggesting that volume rendering replace viewing of multiplanar reconstructions for diagnosis, but merely evaluating the 2 viewing modes objectively. In addition, there were few type 1 errors, which may have been a bias of the exclusive volume rendering technique, as this was the case for monoscopy and stereoscopy. In other words, with volume rendering, observers had no difficulty in recognizing normal vessels.
We split the cases into 2 sessions, so that cases would be shown to a participant as either monoscopy first and then stereoscopy or vice versa, resulting in a form of internal control. A limitation of this method was the need to show the same images and cases to the participants in the second session, but in the alternate viewing mode, with the possibility of participants remembering cases. The second session was performed at least 2 weeks after the first session to decrease bias from the participants remembering a given case from the first session. The participants were not informed that the cases were the same between sessions. We also introduced one non-test case into each set of cases, so that only 9 out of 10 cases in a test batch were scored in an attempt to decrease the participants' ability to recall previous cases.
Time spent per case was not significantly different for monoscopic and stereoscopic display. It might be expected that time would be shorter for stereoscopy however, a similar study by Harake et al noted that participants spent longer looking at stereoscopic images as they perceived that there was more information available [13].
User interaction in this study was limited to rotation, panning, and zooming. The ability to move and rotate the image has been shown to enhance perception of 3D nonmedical objects regardless of whether monoscopic and stereoscopic displays were used, with use of rotation and monoscopic display being superior to static stereoscopy along [30]. The effect of manipulation on stereoscopy is difficult to isolate in our study, however, subjects were allowed equal freedom to manipulate the object for both display modes in our study. Therefore, monoscopic display acted as a control for stereoscopic display.
An additional limitation of the study was that the results are not generalizable to other forms of congenital heart disease or other pathology. However, the study did compare monoscopic display to 3D stereoscopic display of complex thoracic anatomy in an objective manner. While the results are similar to other studies, additional objective studies are required to determine the role of stereoscopic display of volume rendered imaging on perception in medical imaging. No cost benefit analysis was performed in this study, and this type of analysis should be performed prior to routinely using this type of technology in medical imaging [9].

Conclusion
We have completed a retrospective blinded observational study to objectively compare the effect of the stereoscopic 3D display to the monoscopic conventional display of volume-rendered CT images in the setting of pediatric pulmonary vein stenosis. There was no significant overall improvement in the identification of lesions when using stereoscopic 3D display, however, 70% of users reported greater subjective confidence, while objectively, users localized pulmonary veins more accurately and displayed improved accuracy for complex multi-lesion cases of pulmonary vein stenosis. Further studies of other diseases would be useful in determining the potential benefit and role of stereoscopic display for medical imaging diagnosis.
Supporting information S1